Monocular stereoscopic imaging device

ABSTRACT

The monocular stereoscopic imaging device according to one aspect of the presently disclosed subject matter includes: an imaging optical system including a zoom lens and a diaphragm; a pupil dividing unit configured to divide a light flux having passed through a imaging optical system into multiple light fluxes; an imaging unit configured to receive the multiple light fluxes, so as to continuously acquire a left-eye image and a right-eye image; and a controlling unit configured to control a zoom lens driving unit to move the zoom lens in accordance with an instruction of changing the focus distance, and configured to control the diaphragm driving unit to maintain at a substantially constant level a stereoscopic effect of the left-eye image and the right-eye image three-dimensionally displayed on a display unit before and after the zoom lens is moved.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a PCT Bypass continuation application and claims thepriority benefit under 35 U.S.C. §120 of PCT Application No.PCT/JP2011/061804 filed on May 24, 2011 which application designates theU.S., and also claims the priority benefit under 35 U.S.C. §119 ofJapanese Patent Application No. 2010-147891 filed on Jun. 29, 2010,which applications are all hereby incorporated by reference in theirentireties.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The presently disclosed subject matter relates to a monocularstereoscopic imaging device, and more particularly to a technique forimaging each image of an object having passed through different areas inthe right and left direction of a taking lens on an image sensor, so asto acquire an image for a left-eye and an image for a right-eye.

2. Description of the Related Art

Japanese Patent Application Laid-Open No. 10-042314 discloses a parallaximage input device capable of photographing a parallax image using asingle taking lens and an image sensor.

In a monocular 3D camera using such a single taking lens, and dividing aluminous flux having passed through the single taking lens into multipleluminous fluxes (pupil division) so as to photograph a parallax image(referred to as a pupil division system, hereinafter), the parallax at afocus position becomes 0, and parallaxes occur at non-focus positionsdepending on the degree of each defocus. When a focus lens is moved, andthe focus point is changed, the parallax is also changed depending onthe degree of its defocus (referred to as a degree of parallax,hereinafter).

SUMMARY OF THE INVENTION

When photographing an image for a three-dimensional display using amonocular 3D camera, a user can monitor how the image has beenphotographed by three-dimensionally displaying an image for a left-eye(referred to as a left-eye image, hereinafter) and an image for aright-eye (referred to as a right-eye image, hereinafter).

In a monocular 3D camera employing the pupil division system, thereoccurs a parallax on an object located at a position out of focus(position out of focus in a normal 2D camera). In the monocular 3Dcamera employing the pupil division system, an object in focus has aparallax of 0. On the other hand, an object located more frontward(closer to the 3D camera) than the object in focus appears to be poppingout from a display plane toward an observer (user), and an objectlocated more backward than the object in focus (farther from the 3Dcamera) appears to be located backward from the display plane to theuser.

Under a condition in which the degree of defocus is increased on the 2Dcamera, the degree of parallax is greater on the monocular 3D camera.When photographing an object located at a position out of focus with a2D camera, a shorter focus distance (wide-angle side) makes the depth offield deeper so that the degree of defocus is decreased; and a longerfocus distance (telephoto side) makes the depth of field shallower sothat the degree of defocus is increased. Specifically, on the monocular3D camera, when a zooming operation is carried out by shifting a zoomlens in the direction of decreasing the focus distance (direction fromthe telephoto end toward the wide-angle end), the parallax in thedivergence direction (direction of retreating from the display plane)and the excessive parallax (direction of popping out from the displayscreen) (referred to as a stereoscopic effect, hereinafter) aredecreased, and the zoom operation in the direction of increasing thefocus distance (direction from the wide-angle end toward the telephotoend) increases the stereoscopic effect.

Consequently, the zoom operation while photographing moving images orlive view images (through images) may cause uncomfortable feeling to aphotographer due to variation in stereoscopic effect. This is a typicalphenomenon of the monocular 3D camera using the pupil division system.

However, Japanese Patent Application Laid-Open No. 10-042314 describesno parallax adjustment.

Japanese Patent Application Laid-Open No. 2010-81010 describes atechnique of a three-dimensional display or a two-dimensional displayusing a smaller parallax during the zoom operation. This is directed toa technique of prevention of variation in the stereoscopic effect due toa difference in motors and mechanisms among various photographing unitsof a multi-lens 3D camera for photographing a parallax image using twoimaging systems. Hence, it is difficult to apply this technique directlyto the monocular 3D camera having a single photographing unit, and thistechnique cannot solve the problems resulted from the typical phenomenonof the monocular 3D camera.

An object of the presently disclosed subject matter, which has been madein view of circumstances described above, is to provide a monocularstereoscopic imaging device capable of maintaining a stereoscopic effectof a left-eye image and a right-eye image that are three-dimensionallydisplayed at a substantially constant level even if a zoom lens is movedwhile photographing through images or moving images using a monocular 3Dcamera, thereby reducing an uncomfortable feeling of a photographer.

A monocular stereoscopic imaging device according to a first aspect ofthe presently disclosed subject matter includes: an imaging opticalsystem including a zoom lens and a diaphragm; a pupil dividing unitconfigured to divide a light flux having passed through the imagingoptical system into multiple light fluxes; an imaging unit configured toreceive the multiple light fluxes divided by the pupil dividing unit, soas to continuously acquire a left-eye image and a right-eye image; azoom lens driving unit configured to move the zoom lens; a diaphragmdriving unit for driving the diaphragm so as to change a degree ofaperture of an aperture of the diaphragm; a display unit configured torecognizably display the left-eye image and the right-eye image as athree-dimensional image; a display controlling unit configured tothree-dimensionally display on the display unit the left-eye image andthe right-eye image continuously acquired; an input unit configured toinput an instruction of changing a focus distance; and a controllingunit configured to control the zoom lens driving unit to move the zoomlens in accordance with the instruction of changing the focus distanceinput from the input unit, and configured to control the diaphragmdriving unit to maintain at a substantially constant level astereoscopic effect of the left-eye image and the right-eye imagethree-dimensionally displayed on the display unit before or after thezoom lens is moved.

The monocular stereoscopic imaging device according to the first aspectmoves the zoom lens in accordance with the instruction of changing thefocus distance, and drives the diaphragm so as to change the degree ofaperture of the aperture, thereby maintaining at a substantiallyconstant level a stereoscopic effect of the left-eye image and theright-eye image three-dimensionally displayed on the display unit beforeand after the zoom lens is moved. Accordingly, even if the zoom lens ismoved while photographing moving images, the stereoscopic effect of theleft-eye image and the right-eye image that are three-dimensionallydisplayed can be maintained at a substantially constant level, therebyreducing an uncomfortable feeling of a photographer. The description,substantially constant, represents a concept including that thestereoscopic effect is slightly varied because the stereoscopic effectcannot be perfectly maintained at a constant level when the drivingsteps of the diaphragm are limited (when the diaphragm cannot becontinuously driven).

The monocular stereoscopic imaging device according to a second aspectof the presently disclosed subject matter is configured such that, inthe first aspect, the controlling unit drives the diaphragm through thediaphragm driving unit so as to reduce the degree of aperture when thezoom lens is moved in a direction of increasing the focus distancethrough the zoom lens driving unit.

In the monocular stereoscopic imaging device according to the secondaspect, when the zoom lens is moved in the direction of increasing thefocus distance, the stereoscopic effect is increased; therefore, thestereoscopic effect is decreased by reducing the degree of aperture ofthe diaphragm. Accordingly, the stereoscopic effect can be maintained ata substantially constant level regardless of the variation of the focusdistance.

The monocular stereoscopic imaging device according to a third aspect ofthe presently disclosed subject matter is configured such that, in thefirst or second aspect, the controlling unit drives the diaphragmthrough the diaphragm driving unit so as to increase the degree ofaperture when the zoom lens is moved in a direction of decreasing thefocus distance through the zoom lens driving unit.

In the monocular stereoscopic imaging device according to the thirdaspect, when the zoom lens is moved in the direction of decreasing thefocus distance, the stereoscopic effect is decreased; therefore, thestereoscopic effect is increased by increasing the degree of aperture ofthe diaphragm. Accordingly, the stereoscopic effect can be maintained ata substantially constant level regardless of the variation of the focusdistance.

The monocular stereoscopic imaging device according to a fourth aspectof the presently disclosed subject matter is configured such that, inany of the first to third aspects, the controlling unit controls thediaphragm driving unit to minimize the degree of aperture when the zoomlens is located at a telephoto end.

In the monocular stereoscopic imaging device according to the fourthaspect, when the zoom lens is located at the telephoto end, the zoomlens is moved only in the direction of decreasing the focus distance;therefore, the diaphragm can be controlled to minimize the degree ofaperture. Accordingly, it is possible to prevent a difficulty inadjustment of the stereoscopic effect through the diaphragm.

The monocular stereoscopic imaging device according to a fifth aspect ofthe presently disclosed subject matter is configured such that, in anyof the first to fourth aspects, the controlling unit controls thediaphragm driving unit to maximize the degree of aperture when the zoomlens is located at a wide-angle end.

In the monocular stereoscopic imaging device according to the fifthaspect, when the zoom lens is located at the wide-angle end, the zoomlens is moved only in the direction of increasing the focus distance;therefore, the diaphragm can be controlled to maximize the degree ofaperture. Accordingly, it is possible to prevent a difficulty inadjustment of the stereoscopic effect through the diaphragm.

The monocular stereoscopic imaging device according to a sixth aspect ofthe presently disclosed subject matter is configured such that, in anyof the first to fifth aspects, the diaphragm can be driven so as tochange the degree of aperture in n steps (n is a natural number equal toor greater than 2); the zoom lens can be driven so as to change thefocus distance in m steps (m is a natural number equal to or greaterthan 2), m being greater than the n; and the controlling unit controlsthe zoom lens driving unit and the diaphragm driving unit to reduce theaperture of the diaphragm by one stop before the zoom lens is moved whenthe instruction of changing the focus distance in the direction ofincreasing the focus distance is input through the input unit, and toreduce the aperture of the diaphragm by one stop every time the zoomlens is moved by a predetermined number of steps.

In the monocular stereoscopic imaging device according to the sixthaspect, when the diaphragm can be driven to change the degree ofaperture in n steps (n is a natural number equal to or greater than 2),and it can be driven to change the focus distance in m steps (m is anatural number equal to or greater than 2) m being greater than the n,that is, when the aperture cannot be continuously varied, and the zoomlens has the number of the driving steps more than the number of thedriving steps of the diaphragm, and when the instruction of changing thefocus distance in the direction of increasing the focus distance, theaperture of the diaphragm is reduced by one stop before the zoom lens ismoved, and this reduction by one stop of the aperture of the diaphragmis executed every time the zoom lens is moved by predetermined stops.When the number of the driving steps of the diaphragm is limited (thediaphragm cannot be driven continuously), it is impossible to perfectlymaintain the stereoscopic effect at a constant level, thereby thestereoscopic effect is slightly varied; however, the zoom lens and thediaphragm are controlled in the above manner, thereby preventing thestereoscopic effect from being greater than the stereoscopic effectbefore the zoom lens is moved. Accordingly, it is possible to reduce anuncomfortable feeling of the photographer to be as small as possible.

The monocular stereoscopic imaging device according to a seventh aspectof the presently disclosed subject matter is configured such that, inthe sixth aspect, the controlling unit limits the driving steps of thezoom lens to be the n steps of the m steps, and controls the zoom lensdriving unit and the diaphragm driving unit to synchronously drive thediaphragm and the zoom lens.

In the monocular stereoscopic imaging device according to the seventhaspect, when the aperture cannot be continuously varied, and the zoomlens has the number of the driving steps more than the number of thedriving steps of the diaphragm, the driving of the zoom lens is limitedsuch that the zoom lens has the number of the driving steps equal to thenumber of the driving steps of the diaphragm, and the zoom lens and thediaphragm are synchronously driven. In this configuration, although thezooming becomes discontinuous, the slight variation of the stereoscopiceffect can be prevented.

The monocular stereoscopic imaging device according to an eighth aspectof the presently disclosed subject matter is configured such that, inthe sixth aspect, the monocular stereoscopic imaging device furtherincludes a digital zoom unit configured to cut off predetermined areasfrom the left-eye image and the right-eye image, and configured toelectronically change the focus distance, and the controlling unitcontrols the digital zoom unit to virtually change the focus distancethrough the digital zoom unit, instead of moving the zoom lens throughthe zoom lens driving unit.

In the monocular stereoscopic imaging device according to the eighthaspect, the focus distance is virtually varied by the digital zoom unitthat cuts off predetermined areas from the left-eye image and theright-eye image, and electronically changing the focus distance.Accordingly, of two factors that affect the variation of thestereoscopic effect: (1) the variation of the stereoscopic effect due tovariation of the incident light flux; and (2) the variation of thestereoscopic effect due to variation of the angle of view, it ispossible to eliminate the factor (1). Consequently, in the case wherethe number of the driving steps of the diaphragm is smaller than that ofthe zoom lens, and thereby the slight variation of the stereoscopiceffect is inevitable, it is possible to reduce an uncomfortable feelingof the photographer.

The monocular stereoscopic imaging device according to a ninth aspect ofthe presently disclosed subject matter is configured such that, in anyof the first to eighth aspects, the monocular stereoscopic imagingdevice further includes a storage unit configured to store a relationbetween the degree of aperture of the aperture of the diaphragm and thefocus distance, and the controlling unit controls the diaphragm drivingunit based on the relation between the degree of aperture and the focusdistance stored on the storage unit.

The monocular stereoscopic imaging device according to a tenth aspect ofthe presently disclosed subject matter is configured such that themonocular stereoscopic imaging device further includes a two-dimensionalimage generating unit configured to synthesize the left-eye image andthe right-eye image when brightness of the left-eye image and theright-eye image is equal to or less than a predetermined value, so as togenerate a two-dimensional image, the display unit can display thetwo-dimensional image, and the display controlling unit displays thegenerated two-dimensional image on the display unit when thetwo-dimensional image is generated by the two-dimensional imagegenerating unit.

In the monocular stereoscopic imaging device according to the tenthaspect, when brightness of the left-eye image and the right-eye image isequal to or less than the predetermined value, that is, when thedisplayed three-dimensional image becomes dark, which deteriorates thevisibility, the left-eye image and the right-eye image are synthesizedso as to generate a two-dimensional image, and display this.Accordingly, it is possible to enhance the visibility. In addition, thestereoscopic effect becomes smaller in the two-dimensional image than inthe three-dimensional image, thereby it is possible to reduce anuncomfortable feeling of the photographer due to the stereoscopic effectbeing increased.

According to the presently disclosed subject matter, it is possible tomaintain, at a substantially constant level, the stereoscopic effect ofthe left-eye image and the right-eye image that are three-dimensionallydisplayed, even if the zoom lens is moved while photographing throughimages or moving images using the monocular 3D camera, thereby reducingan uncomfortable feeling of a photographer.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a front perspective view of a monocular stereoscopic imagingdevice 1 according to a first embodiment of the presently disclosedsubject matter;

FIG. 2 is a rear perspective view of the monocular stereoscopic imagingdevice 1;

FIG. 3A is a drawing illustrating an example of a configuration of aphase-difference CCD of the monocular stereoscopic imaging device 1;

FIG. 3B is a drawing illustrating an example of the configuration of thephase-difference CCD of the monocular stereoscopic imaging device 1(main pixel);

FIG. 3C is a drawing illustrating an example of the configuration of thephase-difference CCD of the monocular stereoscopic imaging device 1(subpixel);

FIG. 4 is a drawing illustrating each pixel of the main pixels andsubpixels of the phase-difference CCD, a taking lens, and a diaphragm;

FIG. 5A is an enlarged view of a main part of FIG. 4 (normal pixel withno pupil division);

FIG. 5B is an enlarged view of a main part of FIG. 4 (phase-differencepixel with pupil division);

FIG. 5C is an enlarged view of a main part of FIG. 4 (phase-differencepixel with pupil division);

FIG. 6A is a drawing illustrating a separated state of an image imagedon an image sensor with a focus point in front of an object;

FIG. 6B is a drawing illustrating a separated state of an image imagedon the image sensor with the focus point corresponding to the subject(best focus);

FIG. 6C is a drawing illustrating a separated state of an image imagedon the image sensor with the focus point in back of the object;

FIG. 7 is a block diagram of an internal configuration of the monocularstereoscopic imaging device 1;

FIG. 8 is a drawing illustrating a relation between a focus distance anda stereoscopic effect;

FIG. 9A is a drawing illustrating the relation between the focusdistance and the stereoscopic effect when zooming is carried out from awide-angle end toward a telephoto end;

FIG. 9B is a drawing illustrating the relation between the focusdistance and the stereoscopic effect when zooming is carried out fromthe telephoto end toward the wide-angle end;

FIG. 10A is a drawing explaining a relation between the degree ofaperture of a diaphragm 16 and a parallax for an image with the focusposition located in back of the object in the case of using a smallerdegree of aperture;

FIG. 10B is a drawing explaining the relation between the degree ofaperture of the diaphragm 16 and the parallax for an image with thefocus position located in back of the object in the case of using agreater degree of the aperture;

FIG. 11 is a flow chart of a photographing process of through images inthe monocular stereoscopic imaging device 1;

FIG. 12 is a drawing illustrating the relation between the focusdistance and the stereoscopic effect, which is used for maintaining thestereoscopic effect at a constant level when the diaphragm has thenumber of driving steps smaller than the number of driving steps of thezoom lens;

FIG. 13 is a flow chart of the photographing process of the throughimages in a monocular stereoscopic imaging device 2;

FIG. 14 is a flow chart of the photographing process of the throughimages in a monocular stereoscopic imaging device 3; and

FIG. 15 is a drawing illustrating the relation between the focusdistance and the stereoscopic effect, which is used for maintaining thestereoscopic effect at a constant level when the diaphragm has thenumber of driving steps smaller than the number of driving steps of thezoom lens.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Hereinafter, description will be provided on embodiments of a monocularstereoscopic imaging device according to the presently disclosed subjectmatter with reference to the accompanying drawings.

First Embodiment

[Outline of Configuration of Imaging Device]

FIG. 1 is a front perspective view of a monocular stereoscopic imagingdevice 1 according to a first embodiment of the presently disclosedsubject matter. FIG. 2 is a rear perspective view of the monocularstereoscopic imaging device 1. The monocular stereoscopic imaging device1 is a digital camera for receiving light having passed through a lenson an image sensor, and converting the light into digital signals, andrecording the signals on a storage media.

A camera body 10 of the monocular stereoscopic imaging device 1 isformed in a laterally long rectangular parallelopiped box shape. Asillustrated in FIG. 1, a lens unit 12, a strobe 21, and other componentsare disposed on the front face of the camera body 10. A shutter button22, a power/mode switch 24, a mode dial 26, and other components aredisposed on the upper face of the camera body 10. In addition, asillustrated in FIG. 2, a liquid crystal monitor 28, a zoom button 30, across button 32, a MENU/OK button 34, a reproducing button 36, a BACKbutton 38, and other components are disposed on the rear face of thecamera body 10.

A tripod hole as well as a battery slot and a memory card slot with acover that can be opened and closed are disposed on the bottom face (notillustrated) of the camera body 10, and a battery and a memory card arerespectively inserted into the battery slot and the memory card slot.

The lens unit 12 includes a retractable zoom lens, which comes out fromthe camera body 10 by setting the mode of the camera to a photographingmode using the power/mode switch 24. The zoom mechanism and theretracting mechanism of the lens unit 12 are well-known techniques; thusdescriptions of their detailed configurations will be omitted.

The strobe 21 irradiates a strobe light toward a major object.

The shutter button 22 is a two-stroke switch including a so-called “halfpress” and “full press.” While the monocular stereoscopic imaging device1 is operating in the photographing mode, an AE/AF is activated by“half-pressing” the shutter button 22, and photographing is executed by“fully pressing” the shutter button 22. While the monocular stereoscopicimaging device 1 is operating in a projecting mode, projecting isexecuted by “fully pressing” the shutter button 22.

The power/mode switch 24 is an operating member having a function as apower switch for powering on and off the monocular stereoscopic imagingdevice 1, as well as a function as a mode switch for setting the mode ofthe monocular stereoscopic imaging device 1. The power/mode switch 24 isdisposed so as to be slidable from an “OFF position,” a “reproducingposition” to a “photographing position,” and vice versa. The monocularstereoscopic imaging device 1 is powered on by sliding the power/modeswitch 24 to the “reproducing position” or to the “photographingposition,” and powered off by sliding power/mode switch 24 to the “OFFposition.” The power/mode switch 24 is slid to the “reproducingposition” so as to be set in the “reproducing mode,” and to the“photographing position” to be set in the “photographing mode.”

The mode dial 26 functions as an operating member dedicated to thephotographing mode setting for setting the photographing mode of themonocular stereoscopic imaging device 1. In accordance with the settingposition of the mode dial 26, the photographing mode of the monocularstereoscopic imaging device 1 is set to various modes. The photographingmode of the monocular stereoscopic imaging device 1 includes a “planeimage photographing mode” for photographing a plane image, a“stereoscopic vision image photographing mode” for photographing astereoscopic vision image (3D image), a “moving image photographingmode” for photographing a moving image, and a “3D panorama photographingmode” for photographing a three-dimensional panorama, etc.

The liquid crystal monitor 28 is a three-dimensional display devicecapable of displaying a stereoscopic vision image (left-eye image andright-eye image) as directional images having respective predetermineddirectivity using a parallax barrier. When the stereoscopic vision imageis input into the liquid crystal monitor 28, the parallax barrierconstituted of patterns of light transparent sections and lightshielding sections arranged alternately with a predetermined interval isgenerated on a parallax barrier display layer of the liquid crystalmonitor 28, and strip image pieces for illustrating the right and leftimages are alternately arranged and displayed on an image display planeof a layer under this parallax barrier layer. When the liquid crystalmonitor 28 is used for a plane image or as a user interface displaypanel, nothing is displayed on the parallax barrier display layer, andan image is displayed as it is on the image display plane of the layerunder the parallax barrier display layer. The configuration of themonitor 28 is not limited to this, and any display devices that allowthe user to view the left-eye mage and the right-eye image separately byusing a lenticular lens, or special glasses such as polarization glassesand liquid crystal shutter glasses may be applicable, as long as thosedisplay devices recognizably display a stereoscopic vision image as athree-dimensional image. An organic EL (electroluminescence) display orthe like may be used instead of the liquid crystal monitor.

The zoom button 30 functions as an operating member dedicated to a zoominstruction for instructing the zoom operation. The zoom button 30includes a zoom telephoto button 30T for instructing a zoom toward atelephoto side, and a zoom wide button 30W for instructing a zoom towarda wide side. In the monocular stereoscopic imaging device 1, the focusdistance of the lens unit 12 is changed by operating the zoom telephotobutton 30T and the zoom wide button 30W in the photographing mode. Animage under reproduction is scaled up or down by operating the zoomtelephoto button 30T and the zoom wide button 30W in the reproducingmode.

The cross button 32 is an operating member for inputting an instructionof the four directions that are upward, downward, leftward and rightwarddirections, and functions as a button for selecting an appropriate itemfrom a menu display screen, or instructing a selection of varioussetting items from each menu (operating member for cursor operation andmovement). The right and left keys function as frame advance (forwarddirection/backward direction frame advance) buttons in the reproducingmode.

The MENU/OK button 34 is an operating key functioning as a menu buttonfor instructing a display of the menu on the display screen of theliquid crystal monitor 28, as well as functioning as an OK button forinstructing a decision and an execution of the selected operation.

The reproducing button 36 is a button for changing over to thereproducing mode so as to display on the liquid crystal monitor 28 stillimages or moving images of stereoscopic vision images (3D images) orplane images (2D images) that have been photographed and recorded.

The BACK button 38 functions as a button for instructing a cancel of aninput operation or return to a previous operational state.

[Configuration Example of Imaging Optical System and Image Sensor]

The lens unit 12 includes a taking lens 14, a diaphragm 16, asolid-state image sensor (referred to as a phase-difference CCD(charge-coupled device), hereinafter) 17 that is a phase-differenceimage sensor.

The taking lens 14 is an imaging optical system constituted of multiplelenses including a zoom lens for changing the focus distance of thetaking lens 14, and a focus lens for adjusting the focus of the takinglens 14. The zoom lens and the focus lens are continuously movable intheir optical axes.

The diaphragm 16 includes an iris diaphragm, for example. The diaphragm16 is configured to have an aperture whose maximum value is F2.8 andwhose minimum value is F11, as one example, so that the degree ofaperture is continuously or gradually changeable between the maximumvalue and the minimum value.

FIG. 3A to FIG. 3C are drawings illustrating an example of theconfiguration of the phase-difference CCD 17.

The phase-difference CCD 17 has pixels of odd lines (main pixels) andpixels of even lines (subpixels) that are each arranged in a matrix, andimage signals for double planes each photoelectrically converted by themain pixels and the subpixels can be read out separately.

As illustrated in FIG. 3B, among pixels having color filters of R (red),G (green), and B (blue), the odd lines (1, 3, 5, . . . ) of thephase-difference CCD 17 are provided with lines of a pixel array of GRGR. . . and lines of a pixel array of BGBG . . . alternately arranged. Onthe other hand, as illustrated in FIG. 3C, as similar to the odd lines,pixels on the even lines (2, 4, 6, . . . ) are provided with lines of apixel array of GRGR . . . and lines of a pixel arrays of BGBG . . .alternately arranged, and each pixel is shifted by ½ pitch in the linedirection relative to the pixels on the even lines.

FIG. 4 is a drawing illustrating the taking lens 14 and each pixel ofthe main pixels and the subpixels of the phase-difference CCD 17, andFIG. 5A to FIG. 5C are enlarged views of the main part of FIG. 4.

A light shielding member 17A is disposed on a front side (a microlens Lside) of the main pixel of the phase-difference CCD 17, and the lightshielding member 17B is disposed on a front side of the subpixelthereof. Each of the light shielding members 17A and 17B has a functionas a pupil dividing member. As illustrated in FIG. 5A, a light fluxpassing through an exit pupil enters a pixel of a normal CCD (photodiodePD) through the microlens L without any limitation. As illustrated inFIG. 5B, the light shielding member 17A shields the right half of alight receiving surface of the main pixel (photodiode PD). Hence, themain pixel receives light only on the left side relative to the opticalaxis of the light flux passing through the exit pupil. As illustrated inFIG. 5C, the light shielding member 17B shields the left half of thelight receiving surface of the subpixel (photodiode PD). Hence, thesubpixel receives light only on the right side relative to the opticalaxis of the light flux passing through the exit pupil.

Hereinafter, description will be provided on a mechanism ofphotographing a stereoscopic vision image on the phase-difference CCD 17in which each main pixel of the phase-difference CCD 17 receives lightonly on the left side relative to the optical axis of the light fluxpassing through the exit pupil, and each subpixel receives light only onthe right side relative to the optical axis of the light flux passingthrough the exit pupil in the above manner.

FIG. 6A to FIG. 6C are drawings illustrating separated states of animage imaged on the image sensor is separated depending on thedifference in the focus point of the focus lens, i.e., a focus point infront of the subject, a focus point corresponding to the subject (bestfocus), and a focus point in back of the subject. The diaphragm 16 isomitted in the FIG. 6A to FIG. 6C for the sake of comparing thedifference in the image separation resulting from the focus point.

As illustrated in FIG. 6B, the pupil-divided images in focus are imagedat the identical position (corresponds to each other) on the imagesensor. On the other hand, as illustrated in FIG. 6A and FIG. 6C, thepupil-divided images focused in front or in back of the subject areimaged at different positions (separated from each other) on the imagesensor 17.

Accordingly, the object images pupil-divided in the left and rightdirections are acquired through the phase-difference CCD 17, therebyacquiring the left-eye image and the right-eye image (stereoscopicvision image) having different parallaxes depending on the focusposition.

In the phase-difference CCD 17 having the above-mentioned configuration,the main pixel and the subpixel are respectively configured to limit thelight fluxes by the light shielding members 17A, 17B on different areas(on the right half, on the left half) from each other, but the presentlydisclosed subject matter is not limited to this. Without providing thelight shielding members 17A, 17B, the microlens L may be shiftedrelative to the photodiode PD in the right and left directions, and thelight flux entering the photodiode PD may be limited depending on thisshifting direction, for example. A single microlens may be provided fortwo pixels (main pixel and subpixel), thereby limiting the light fluxentering each pixel.

[Internal Configuration of Imaging Device]

FIG. 7 is a block diagram of the monocular stereoscopic imaging device 1according to the first embodiment of the presently disclosed subjectmatter. The monocular stereoscopic imaging device 1 records an acquiredimage on a recording media 54, and the overall operation thereof iscomprehensively controlled by a central processing device (CPU; centralprocessing unit) 40.

The monocular stereoscopic imaging device 1 has an operating unit 48including the shutter button, the mode dial, the reproducing button, aMENU/OK button, a cross button, a BACK button, and other components.Signals from the operating unit 48 are input into the CPU 40. The CPU 40controls each circuit of the monocular stereoscopic imaging device 1based on the input signals from the operating unit 48, so as to executea lens driving control, a diaphragm driving control, a photographingoperation control, an image processing control, an image datarecording/reproducing control, a display control of the liquid crystalmonitor 28 for the three-dimensional display.

When the stereoscopic imaging device 1 is powered on by using thepower/mode switch 24, the power is fed to each block from a power source58, and then the stereoscopic imaging device 1 is activated.

The light flux having passed through the taking lens 14, the diaphragm16, and others is imaged on the phase-difference CCD 17, and signalcharges are accumulated in the phase-difference CCD 17. The signalcharges accumulated in the phase-difference CCD 17 are read out asvoltage signals corresponding to the signal charges in accordance withreading signals provided from a timing generator 45. The voltage signalsread out from the phase-difference CCD 17 are input into an analoguesignal processing unit 60.

The analogue signal processing unit 60 carries out correlative doublesampling processing on the voltage signals output from thephase-difference CCD 17 (processing to obtain accurate pixel data byfinding a difference between a field through component level and a pixelsignal component level contained in an output signal for each pixel ofthe image sensor, so as to reduce noises (particularly, thermal noises)contained in the output signals of the image sensor), therebysampling-holding the R, G, B signals for each pixel. These voltagesignals are amplified, and then input into an AD converter 61. The ADconverter 61 converts the R, G, B signals that are sequentially inputinto digital R, G, B signals, and then outputs the signals to an imageinput controller 62.

A digital signal processing unit 63 executes predetermined signalprocessing such as offset processing, gain control processing includinga white balance correction and a sensitivity correction, gammacorrection processing, and YC processing on the digital image signalsinput through the image input controller 62. Main pixel data read outfrom the main pixels on the odd lines of the phase-difference CCD 17 areprocessed as left-eye image data, and subpixel data read out from thesubpixels on the even lines are processed as right-eye image data.

The left-eye image data and the right-eye image data (3D image data)processed on the digital signal processing unit 63 are input into a VRAM(Video Random Access Memory) 50. The VRAM 50 includes an area A and anarea B each for storing the 3D image data representing the 3D image ofone frame. The 3D image data representing the 3D image of one frame isalternately overwritten in the area A and in the area B of the VRAM 50.Of the area A and the area B of the VRAM 50, the 3D image data beingwritten is read out from the area other than the area in which the 3Dimage data is being overwritten.

The 3D image data read out from the VRAM 50 is processed on a 3D imagesignal processing unit 64 into strip image pieces. The 3D image dataprocessed into strip image pieces is encoded on a video encoder 66, andoutput to the liquid crystal monitor (3D image display device) 28 forthe three-dimensional display that is disposed on the rear face of thecamera. Accordingly, the 3D object image is continuously displayed onthe display screen of the liquid crystal monitor 28.

The photographer monitors the images (through images) displayed on theliquid crystal monitor 28 in real time so as to confirm the angle ofview of the photographing. In response to the operations of the zoomtelephoto button 30T and the zoom wide button 30W, the CPU 40 moves thezoom lens along the optical axis through the lens driving unit 27, so asto change the focus distance.

FIG. 8 is a graph illustrating a relation between the focus distance andthe stereoscopic effect. The 3D object image is displayed on the liquidcrystal monitor 28 such that the object located at the focus position isdisplayed as if it is located on the screen plane where the parallax is0; the object located backward from the focus position (farther from thephase-difference CCD17) is displayed as if it retreats from the displayplane; and the object located frontward from the focus position (closerto the phase-difference CCD17) is displayed as if it pops out from thedisplay plane. A sum of the parallax in the direction of retracting fromthe display plane (parallax in the divergence direction) and theparallax in the direction of popping out from the display plane(excessive parallax) is defined as the stereoscopic effect.

As illustrated in FIG. 8, the stereoscopic effect varies depending onthe focus distance. The stereoscopic effect is increased as the focusdistance is greater, that is, as the zoom position is shifted to thetelephoto side. On the other hand, the stereoscopic effect is decreasedas the focus distance is smaller, that is, as the zoom position isshifted to the wide-angle side. When the zoom button 30 is operatedwhile displaying through images or moving images on the liquid crystalmonitor 28, the stereoscopic effect varies depending on the focusdistance, which causes an uncomfortable feeling to the photographer.

Therefore, as illustrated in FIG. 9A and FIG. 9B, the CPU 40 controlsthe degree of aperture of the diaphragm 16 in accordance with thevariation of the focus distance, thereby stabilizing the stereoscopiceffect at a constant level around the focus distance. FIG. 10A and FIG.10B are drawings explaining a relation between the degree of aperture ofthe diaphragm 16 and the parallax with the focus position located inback of the object. The degree of aperture is small in FIG. 10A, and thedegree of aperture is large in FIG. 10B. As explained in FIG. 6C, theimages with the focus position located in back of the object are imagedat different positions (separated) on the image sensor, and a smallerdegree of aperture illustrated in FIG. 10A decreases the separation ofthe images, and a greater degree of aperture illustrated in FIG. 10Bincreases the separation of the images. This means that the parallaxbecomes smaller when the degree of aperture of the diaphragm 16 issmaller, and the parallax becomes greater when the degree of aperture ofthe diaphragm 16 is greater.

That is, as illustrated in FIG. 9A, since the stereoscopic effectbecomes greater when the zoom lens is moved from the wide-angle endtoward the telephoto end (in the direction of increasing the focusdistance), the CPU 40 decreases the degree of aperture of the diaphragm16 so as to decrease the stereoscopic effect. Accordingly, thestereoscopic effect can be constantly maintained as the stereoscopiceffect is located at the wide-angle end. As illustrated in FIG. 9B, thestereoscopic effect becomes smaller when the zoom lens is moved from thetelephoto end toward the wide-angle end (in the direction of decreasingthe focus distance), therefore, the CPU 40 increases the degree ofaperture of the diaphragm 16 so as to increase the stereoscopic effect.Accordingly, the stereoscopic effect can be constantly maintained as thestereoscopic effect is located at the telephoto end.

The relation between the degree of aperture of the diaphragm 16 and thefocus distance is stored on a storage area of the CPU 40. More than onerelation is stored there in accordance with the brightness of theobject. The CPU 40 acquires the size of the aperture of the diaphragm 16from the 3D image data, and determines which relation should be used.The CPU 40 varies the degree of the aperture of the diaphragm 16 throughthe diaphragm driving unit 46 based on the driving amount of the zoomlens, that is, the amount of change in the focus distance, and therelation between the determined degree of the aperture and the focusdistance, so as to maintain the stereoscopic effect of thethree-dimensional image displayed on the liquid crystal monitor 28 at aconstant level.

It may be impossible to adjust the stereoscopic effect through thediaphragm 16 depending on the brightness of the object. For example,this may occur when the brightness of the object is great, and the zoomlens is not located at the wide-angle end. In this case, the CPU 40 mayopen the aperture of the diaphragm 16 at a level where the stereoscopiceffect can be adjusted, and thereafter, may adjust the stereoscopiceffect. Specifically, the CPU 40 determines to use a relation in thecase where the aperture of the diaphragm 16 is greater, among therelations between the degree of the aperture and the focus distance,which are stored on the storage area of the CPU 40. The CPU 40 acquiresthe degree of the aperture of the diaphragm 16 corresponding to thecurrent focus distance based on the determined relation, and increasesthe degree of aperture of the aperture until it becomes equal to thisacquired degree of the aperture. Subsequently, the CPU 40 may drive theaperture of the diaphragm 16 in accordance with the movement of the zoomlens using the determined relation. In this case, the aperture of thediaphragm 16 becomes the greatest at the wide-angle end (F11 in thepresent embodiment).

There may be such a case in which the brightness of the object is dark,and the zoom lens is not located at the telephoto end. In this case, theCPU 40 may close the aperture of the diaphragm 16 at a level where thestereoscopic effect can be adjusted, and thereafter, adjust thestereoscopic effect. Specifically, the CPU 40 determines to use arelation in the case where the aperture of the diaphragm 16 is smaller,among the relations between the degree of the aperture and the focusdistance, which are stored on the storage area of the CPU 40. The CPU 40acquires the degree of the aperture of the diaphragm 16 corresponding tothe current focus distance based on the determined relation, anddecreases the degree of aperture of the aperture until it becomes equalto this acquired degree of the aperture. Subsequently, the CPU 40 maydrive the aperture of the diaphragm 16 in accordance with the movementof the zoom lens using the determined relation. In this case, theaperture of the diaphragm 16 becomes the smallest at the telephoto end(F2.8 in the present embodiment).

With reference to FIG. 7 once again, in response to the first stagepressing (half-pressing) of the shutter button 22 of the operating unit48, the CPU 40 starts the AF (automatic focus adjustment) operation andthe AE (automatic exposure) operation, and controls the focus lens tomove in the optical axis direction through the lens driving unit 47 soas to shift the focus lens at the focus position.

An AF processing unit 42 is a part for executing contrast AF processingor phase-difference AF processing. When executing the contrast AFprocessing, the AF processing unit 42 extracts high frequency componentsof the image data stored in a predetermined focus area, from at leastone of the left-eye image data and the right-eye image data. The AFprocessing unit 42 then integrates the high frequency components so asto calculate the AF evaluation values indicating the focus states. AFcontrol is executed by controlling the focus lens in the taking lens 14such that the AF evaluation value becomes a maximum. When executing thephase-difference AF processing, the AF processing unit 42 detects thephase differences between the image data corresponding to the mainpixels and the image data corresponding to the subpixels that arelocated in the predetermined focus area of the left-eye image data andthe right-eye image data, and finds the degree of defocus based on theinformation indicating the detected phase differences. The AF control isexecuted by controlling the focus lens in the taking lens 14 such thatthe degree of defocus becomes 0.

The AF operation is executed not only when the shutter button 22 ispressed at the first stage (half-pressing), but also when the right-eyeimage data and left-eye image data are continuously photographed. Thecontinuous photographing of the right-eye image data and the left-eyeimage data may be conducted in the case of photographing live viewimages (through images), or in the case of photographing moving images,etc. In this case, while continuously photographing the right-eye imagedata and the left-eye image data, the AF processing unit 42 performs thecontinuous AF to continuously control the position of the focus lens byrepetitively executing the calculation of the AF evaluation values allthe time.

The CPU 40 moves the zoom lens frontward and backward in the opticalaxis direction through the lens driving unit 47 if necessary, so as tochange the focus distance.

The image data output from the AD converter 61 at the time ofhalf-pressing the shutter button 22 is captured into an AE/AWB detectingunit 44.

The AE/AWB detecting unit 44 integrates G signals of the entire displayplane, or integrates the G signals having different weights at thecenter portion of the display plane from the surrounding portion of thedisplay plane, and outputs these integrated values to the CPU 40. TheCPU 40 calculates a brightness of the object (photographing Ev value)based on the integrated values input from the AE/AWB detecting unit 44,and based on the photographing Ev value, the CPU 40 determines theaperture value of the diaphragm 16 and the electronic shutter (shutterspeed) of the phase-difference CCD 17 in accordance with a predeterminedprogram diagram. The CPU 40 controls the diaphragm 16 through thediaphragm driving unit 46 based on the determined aperture value, andalso controls the charge accumulating time on the phase-difference CCD17 through the timing generator 45 based on the determined shutterspeed.

After the AE operation and the AF operation are completed, in responseto the second stage pressing (full-pressing) of the shutter button 22,the image data for two images of the left-eye image (main image) and theright-eye image (sub-image) corresponding to the main pixels and thesubpixels output from the AD converter 61 are input from the image inputcontroller 62 into the VRAM 50, and are temporarily stored there.

The image data for two images temporarily stored on the VRAM 50 is readout by the digital signal processing unit 63 at an appropriate timing,and on this digital signal processing unit 63, a predetermined signalprocessing including processing to generate brightness data and colordifference data for the image data (YC processing) is carried out. Theimage data (YC data) to which the YC processing has been carried out isstored on the VRAM 50 once again. Subsequently, the YC data for twoimages are output into a compressing-decompressing unit 65, where apredetermined compression processing such as JPEG (joint photographicexperts group) is carried out to the YC data, and then is stored on theVRAM 50 once again.

From the YC data (compressed data) for two images stored on the VRAM 50,a multi-picture file (MP file: file in a format of combining multipleimages) is generated on the 3D image signal processing unit 64. This MPfile is read out by a media controller (media recording controllingunit) 52, and is recorded on the recording media 54.

The monocular stereoscopic imaging device 1 can record and reproduce notonly moving images and still images but also audios. A microphone 57receives input of external audios. A speaker 56 outputs recorded audios.When recording audios, an audio input and output processing circuit 55encodes the audios input from the microphone 57, and when reproducingthe recorded audios, the audio input and output processing circuit 55decodes the recorded audios, and then outputs the audios to the speaker56.

[Description of Operation of Imaging Device]

The operation of the monocular stereoscopic imaging device 1 will be nowdescribed. This imaging processing is controlled by the CPU 40. Aprogram for allowing the CPU 40 to execute this imaging processing isstored on a program storage unit in the CPU 40.

FIG. 11 is a flow chart illustrating a flow of the photographing anddisplaying operation of the live view images.

In response to an input of an instruction for staring photographing(step S10), the CPU 40 drives the taking lens 14 and the diaphragm 16 tomove to an initial position, so that light of the object having passedthrough the taking lens 14 is imaged through the diaphragm 16 on thelight receiving surface of the phase-difference CCD 17. The signalcharges accumulated on the main pixels and the subpixels of thephase-difference CCD 17 are read out by turns, as the voltage signals(image signals) corresponding to the signal charges, at a predeterminedframe rate in accordance with the timing signals output from the timinggenerator 45. The voltage signals read out are input into the digitalsignal processing unit 63 by turns through the analogue signalprocessing unit 60, the AD converter 61, and the image input controller62, so as to generate the left-eye image data and the right-eye imagedata. The generated left-eye image data and the right-eye image areinput into the VRAM 50 by turns. The left-eye image data and theright-eye image are read out from the VRAM 50 by turns, and the 3D imagesignal processing unit 64 generates brightness/color difference signalsfor the left-eye image data and the right-eye image that have been readout, and the generated brightness/color difference signals are output tothe liquid crystal monitor 28 through the video encoder 66. The parallaxbarrier is displayed on the liquid crystal monitor 28, and the stripimage pieces of the left-eye image data and the right-eye mage data arealternately arranged and displayed on the image display plane of thelayer under this parallax barrier layer (step S11). Accordingly, the 3Dthrough images are displayed on the liquid crystal monitor 28.

The CPU 40 determines whether or not the zoom operation has been carriedout, that is, the instruction for the zoom operation has been inputthrough the operating unit 48 (step S12).

When the zoom operation has been carried out (YES in step S12), the CPU40 moves the zoom lens through the lens driving unit 47 in accordancewith the input from the operating unit 48, so as to change the focusdistance of the taking lens 14. The CPU 40 finds the brightness of theobject based on the left-eye image data and the right-eye image data,and the CPU 40 selects a relation corresponding to the desiredbrightness of the object among the relations between the focus distanceand the degree of the aperture, which are stored on the storage area ofthe CPU 40. The CPU 40 drives the diaphragm 16 through the diaphragmdriving unit 46, and changes the degree of the aperture based on theselected relation, such that the stereoscopic effect before the zoomoperation is carried out (step S12) is maintained (step S13).

When the zoom operation has been executed, the CPU 40 moves the focuslens through the lens driving unit 47 so that the focus position isprevented from being varied along with the movement of the zoom lens.Accordingly, no variation of the parallax occurs even if the focusposition is changed.

After the degree of the aperture of the diaphragm 16 is varied inaccordance with the zoom operation (step S13), or when the zoomoperation has not been carried out (NO in step S12), the CPU 40determines whether or not an instruction for completing thephotographing has been input (step S14).

When no instruction for completing the photographing has been input (NOin step S14), the processes of step S11 to step S14 are carried out onceagain.

When the instruction for completing the photographing (such as thehalf-pressing of the shutter button, the operation to complete thephotographing mode, the operation to power off the power source, etc.)has been input (YES in step S14), the photographing process of thethrough images is completed.

When the shutter button is half-pressed, an S1ON signal is input intothe CPU 40, and the CPU 40 executes the AE/AF operation through the AFprocessing unit 42 and the AE/AWB detecting unit 44. In thephotographing process of a stereoscopic vision image, the AF processingunit 42 executes the AF operation through the phase-difference AFprocessing.

When the shutter button is full-pressed, an S2ON signal is input intothe CPU 40, and the CPU 40 starts the photographing and recordingprocessing. Specifically, the phase-difference CCD 17 is exposed usingthe shutter speed and the aperture value determined based on aphotometry result.

The image data for two images output from the main pixels and thesubpixels of the phase-difference CCD 17 are captured through theanalogue signal processing unit 60, the AD converter 61, and the imageinput controller 62 into the VRAM 50, and are converted into thebrightness/color difference signals on the 3D image signal processingunit 64, and then are stored on the VRAM 50. The left-eye image datastored on the VRAM 50 is added into the compressing-decompressing unit65 so as to be compressed in a predetermined compressed format (JPEGformat, for example), and thereafter the compressed data is stored onthe VRAM 50.

An MP file is generated from the compressed data for two images that isstored on the VRAM 50, and this MP file is recorded on the recordingmedia 54 through the media controller 52. In such a manner, thestereoscopic vision image is photographed and recorded.

The present embodiment has been described by using the example ofphotographing the stereoscopic vision image, but the monocularstereoscopic imaging device 1 can photograph both a plane image and astereoscopic vision image. When photographing of the plane image, onlythe main pixels of the phase-difference CCD 17 may be used forphotographing. The details of the photographing process for the planeimage are the same as those for the stereoscopic vision image; thusdescription thereof will be omitted.

The images recorded on the recording media 54 in this manner can bedisplayed on the liquid crystal monitor 28 by setting the mode of themonocular stereoscopic imaging device 1 to the reproducing mode throughthe reproducing button.

When the mode is set to the reproducing mode, the CPU 40 outputs acommand to the media controller 52 to read out an image file latestrecorded on the recording media 54.

The compressed image data of the image file read out is added into thecompressing-decompressing unit 65, so as to be decompressed into theuncompressed brightness/color difference signals, and thereafter thesignals are output to the liquid crystal monitor 28 through the videoencoder 66.

The frame advance of the images is executed by operating the right andleft keys of the cross key. Pressing the right key of the cross keyenables reading out of a next image file from the recording media 54,and displaying of this image on the liquid crystal monitor 28; andpressing the left key of the cross key enables reading out of a previousfile from the recording media 54, and displaying of this image on theliquid crystal monitor 28.

According to the present embodiment, even if the zoom lens is movedduring the display of the live view images, the stereoscopic effect ofthe three-dimensional images displayed on the liquid crystal monitor 28can be maintained at a constant level all the time, resulting inreduction of the uncomfortable feeling of the photographer.

The present embodiment has been described by using the example of thephotographing and displaying of the live view images, but thisembodiment may be applicable to the case of continuously acquiring theright-eye image data and the left-eye image data, such as the case ofphotographing moving images. The photographing of the live view imagesand the photographing of the moving images are different only in thatthe right-eye image data and the left-eye image data that havecontinuously been photographed are not recorded in the case of the liveview image photographing, whereas the right-eye image data and theleft-eye image data that have continuously been photographed arerecorded on the recording media 54 in addition to the process of FIG. 11in the case of the moving image photographing. The processing ofrecording the right-eye image data and the left-eye image data that havecontinuously been photographed on the recording media 54 is a well-knowntechnique; thus the detailed description thereof will be omitted.

In the present embodiment, a graph of the relation between the amount ofchange in degree of the aperture of the diaphragm 16 and the amount ofchange in focus distance is stored on the storage area of the CPU 40,and the CPU 40 changes the degree of the aperture of the diaphragm 16based on this stored relation and the amount of the zoom lens driving,but the method of changing the degree of the aperture of the diaphragm16 is not limited to this. For example, a maximum value of the parallaxin the divergence direction and a maximum value of the excessiveparallax may be obtained from the right-eye image data and the left-eyeimage data before the zoom lens is moved, and the maximum value of theparallax in the divergence direction and the maximum value of theexcessive parallax may be obtained in real time from the right-eye imagedata and the left-eye image data that are varied along with the movementof the zoom lens, and the aperture of the diaphragm 16 may be variedsuch that the maximum value of the parallax in the divergence directionand the maximum value of the excessive parallax can be constantlymaintained. In this case, no information is needed to be stored on thestorage area of the CPU 40. The relation between the amount of change indegree of the aperture of the diaphragm 16 and the amount of change infocus distance that are stored on the storage area of the CPU 40 is notlimited to a graph form illustrated in FIG. 9 and others. For example, acorresponding map among the position of the zoom lens, the degree ofaperture of the aperture of the diaphragm 16, and the maximum values ofthe parallax in the divergence direction and the excessive parallax, orthe like may also be applicable.

In the present embodiment, the diaphragm 16 is driven in accordance withthe movement of the zoom lens such that the stereoscopic effect that isthe sum of the parallax in the divergence direction and the excessiveparallax can be constant, and the excessive parallax greatly affects thevisibility of the stereoscopic effect. Hence, the diaphragm 16 may bedriven in accordance with the movement of the zoom lens such that themaximum value of the excessive parallax becomes substantially constant,thereby maintaining the stereoscopic effect at a substantially constantlevel.

The parallax of a major object (such as a human face and an object atthe center of the display plane) also affects the visibility of thestereoscopic effect. For this reason, the diaphragm 16 may be driven inaccordance with the movement of the zoom lens such that the parallax ofthe major object becomes substantially constant, thereby maintaining thestereoscopic effect at a substantially constant level.

In the present embodiment, the diaphragm 16 has been configured suchthat the aperture value can be continuously varied in a range from F2.8to F11, but another diaphragm having an aperture incapable ofcontinuously changing such as a diaphragm except an iris diaphragm mayalso be used. For example, such a case may be applicable that employs anaperture variable in steps and a zoom lens variable seamlessly(continuously). This case indicates that the zoom lens has more drivingsteps than the driving steps of the diaphragm.

FIG. 12 is a graph illustrating a relation between the focus distanceand the stereoscopic effect in the case where a diaphragm 16′ having anaperture value in a range from F2.8 to F11 and variable in ⅓ EV steps(as an example), in which the stereoscopic effect is varied in fivesteps, is used. FIG. 12 illustrates the variation of the stereoscopiceffect when the zoom lens is moved from the wide-angle end toward thetelephoto end (in the direction of increasing the focus distance). Inthis case, the aperture of the diaphragm cannot be varied continuously,which is different from the case of FIG. 9.

Therefore, the CPU 40 reduces the aperture of the diaphragm 16′ by onestop in a state where the zoom lens is located at the wide-angle end(i.e., the degree of the aperture is reduced by one stop), so as toslightly decrease the stereoscopic effect. The CPU 40 also reduces theaperture of the diaphragm 16′ by one stop before the zoom lens is movedin a case where the zoom lens is not located at the wide-angle end.

Thereafter, as the zoom lens is moved in the telephoto direction (in thedirection of increasing the focus distance), the stereoscopic effect isgradually increased along with the movement of the zoom lens. When thestereoscopic effect becomes equal to the stereoscopic effect in the casewhere the zoom lens is located at the wide-angle end, the CPU 40 reducesthe aperture of the diaphragm 16′ by another one stop. Accordingly, thestereoscopic effect slightly becomes decreased. This operation isrepetitively executed, thereby changing the stereoscopic effect insteps, and maintaining the stereoscopic effect at a substantiallyconstant level.

The focus distance when the aperture of the diaphragm 16′ is reduced byone stop may be stored on the storage area of the CPU 40, and the CPU 40may change the degree of the aperture of the diaphragm based on thisfocus distance. The CPU 40 may acquire the stereoscopic effect from theleft-eye image data and the right-eye image data in real time, and maychange the degree of the aperture of the diaphragm 16′ when thestereoscopic effect becomes equal to that before the zoom lens is moved.

In this configuration, it is possible to maintain the stereoscopiceffect at a substantially constant level (including slight variations).Even if slightly varied, the stereoscopic effect is never varied greaterthan the stereoscopic effect before the zoom lens is moved. Accordingly,an uncomfortable feeling perceived by the photographer can be as smallas possible.

When the zoom lens has been moved from the telephoto end to thewide-angle end (in the direction of reducing the focus distance), it maybe configured such that the zoom lens is moved alone without driving thediaphragm 16′, and when the stereoscopic effect is slightly decreased,the aperture of the diaphragm 16′ is increased by one stop (i.e., thedegree of the aperture is increased by one stop), thereby thestereoscopic effect corresponds to the stereoscopic effect before thezoom lens is moved.

The “slight amount” which is varied in the case where the stereoscopiceffect is maintained at a substantially constant level is resulted fromthe number of the driving steps of the diaphragm used in the adjustmentof the stereoscopic effect. This means that determining the number ofthe driving steps of the diaphragm leads to determining the slightamount varied, and determining the slight amount varied leads todetermining the number of the driving steps of the diaphragm. For thisreason, the number of the driving steps of the diaphragm or the slightamount varied should be set on the storage area of the CPU 40, inadvance.

In the case of using a diaphragm 16′ whose aperture cannot be variedcontinuously, the driving of the zoom lens may be limited such that thezoom lens has the number of the driving steps equal to the number of thedriving steps of the diaphragm 16′. In this case, the driving of thezoom lens should be synchronized with the driving of the diaphragm 16′.The zooming thus becomes discontinuously, but the slight variation ofthe stereoscopic effect can be prevented.

In the present embodiment, the zoom lens continuously movable has beenused, but a zoom lens movable in steps may be used instead.

Second Embodiment

In the first embodiment of the presently disclosed subject matter, thestereoscopic effect of the three-dimensional image displayed on theliquid crystal monitor 28 is constantly maintained all the time even ifthe zoom lens is moved while displaying the live view images, however,the image displayed on the liquid crystal monitor 28 may become darkthrough the adjustment of the stereoscopic effect, which may deterioratethe visibility.

In a second embodiment of the presently disclosed subject matter, whenthe brightness of the displayed image, that is, the brightness of thephotographed image becomes dark, the image is displayed as the 2D image,thereby reducing an uncomfortable feeling due to the displayed imagebeing darkened. Hereinafter, description will be provided on a monocularstereoscopic imaging device 2 according to the second embodiment. Theoverall configuration of the imaging device is same as that of the firstembodiment; therefore, description thereof will be omitted, and only theoperation of the imaging device will now be described. In thedescription of the operation of the imaging device, the same elements asthose of the first embodiment are referred to by the same referencenumerals and characters, and description thereof will be omitted.

[Description of Operation of Imaging Device]

Hereinafter, description will be provided on the operation of themonocular stereoscopic imaging device 2. This imaging processing iscontrolled by the CPU 40. A program for allowing the CPU 40 to executethis imaging processing is stored on the program storage unit in the CPU40.

FIG. 13 is a flow chart illustrating a flow of the photographing anddisplaying processing of the live view images.

In response to an input of an instruction for staring the photographing(step S10), the CPU 40 drives the taking lens 14 and the diaphragm 16 tomove to an initial position, so that the light of the object havingpassed through the taking lens 14 is imaged through the diaphragm 16 onthe light receiving surface of the phase-difference CCD 17, and theleft-eye image data and the right-eye image data are generated by turns.Brightness/color difference signals are generated from the abovegenerated left-eye image data and the right-eye image, and the generatedsignals are output to the liquid crystal monitor 28 through the videoencoder 66. In this manner, the 3D through images are displayed on theliquid crystal monitor 28 (step S11).

The CPU 40 determines whether or not the zoom operation has been carriedout, that is, the instruction for the zoom operation has been inputthrough the operating unit 48 (step S12).

When the zoom operation has been carried out (YES in step S12), the CPU40 moves the zoom lens through the lens driving unit 47, so as to changethe focus distance of the taking lens 14 in accordance with the inputfrom the operating unit 48. In order to maintain the stereoscopic effectbefore the zoom operation is carried out (step S12), the CPU 40 drivesthe diaphragm 16 through the diaphragm driving unit 46 based on therelation corresponding to the brightness of the object among therelations between the focus distance and the degree of the aperture,which are stored on the storage area of the CPU 40, and changes thedegree of the aperture (step S13).

After the degree of the aperture of the diaphragm 16 is varied (stepS13) in accordance with the zoom operation, and when the zoom operationhas not been carried out (NO in step S12), the CPU 40 acquires from theAE/AWB detecting unit 44 the integrated value of the G signals of theentire display plane, or the integrated value of the G signals havingdifferent weights between the center portion of the display plane andthe surrounding portion of the display plane. The CPU 40 calculates thebrightness of the object (photographing Ev value) based on the acquiredintegrated values, and determines whether or not the calculatedbrightness of the object is smaller than a predetermined threshold value(step S20). A value indicating a lower limit of the brightness of 3Dthrough images which deteriorates the visibility of the image is storedas the threshold value on the storage area of the CPU 40 in advance. TheCPU 40 executes the above determination by comparing the photographingEv value and the threshold value stored on the storage area. Thephotographing Ev value may be smaller than the threshold value when thedegree of the aperture of the diaphragm 16 is decreased more thanrequired for the brightness of the object, that is, for example, whenthe change in the degree of the aperture of the diaphragm 16 becomesgreater although the photographing target is bright, or when thephotographing target is dark although the change in the degree of theaperture of the diaphragm 16 is not so great.

When determining that the photographing Ev value is not equal to or lessthan the predetermined threshold value (NO in step S20), the CPU 40continues to display the 3D through images in step S11 (step S22).

When determining that the photographing Ev value is equal to or lessthan the predetermined threshold value, the CPU 40 mixes pixels of theleft-eye image data and the right-eye image data obtained from the mainpixels and the subpixels so as to generate a single 2D image data;thereby obtaining bright 2D image data. The generated 2D image data isinput into the VRAM 50 by turns. The 2D image data is read out from theVRAM 50 by turns, and the 3D image signal processing unit 64 generatesthe brightness/color difference signals for the 2D image data that hasbeen read out. The brightness/color difference signals are output to theliquid crystal monitor 28 through the video encoder 66 (step S21). Inthis manner, the 2D through images are displayed on the liquid crystalmonitor 28. At this time, no parallax barrier is displayed on the liquidcrystal monitor 28.

In step S21, the degree of aperture immediately before the 2D throughimages are displayed may be retained as the degree of aperture of theaperture of the diaphragm 16, or the degree of aperture may be varied soas to obtain an appropriate exposure.

The CPU 40 determines whether or not an instruction for completing thephotographing has been input (step S14). When no instruction forcompleting the photographing has been input (No in step S14), the CPU 40executes the processes of step S12 to step S22 once again. When theinstruction for completing the photographing has been input (YES in stepS14), the CPU 40 completes the process of photographing the throughimages. The process for the case in which S1 and S2 are switched ON issame as that in the first embodiment, and therefore, description thereofwill be omitted.

According to the present embodiment, the 2D image is displayed when theimage for display is dark and its visibility is deteriorated, therebyenhancing the visibility. The stereoscopic effect becomes smaller in the2D image than in the 3D image, thereby it is possible to reduce anuncomfortable feeling of the photographer due to the stereoscopic effectbeing increased.

The present embodiment has been described by using the example of thephotographing and displaying of the live view images, but thisembodiment may be applicable to the case of continuously acquiring theright-eye image data and the left-eye image data, such as the case ofphotographing moving images, as similar to the first embodiment.

Third Embodiment

The variation of the first embodiment of the presently disclosed subjectmatter has been described by using the example in which the diaphragm16′ whose aperture cannot be varied continuously, thereby the number ofthe driving steps of the diaphragm becomes smaller than the number ofthe driving steps of the zoom lens, and the slight variation of thestereoscopic effect is inevitable. In this case, the following operationis repetitively executed: when the zoom lens is moved in the telephotodirection (direction of increasing the focus distance), the aperture ofthe diaphragm is reduced by one stop before the zoom lens is moved, andthereafter the zoom lens is moved, so as to maintain the stereoscopiceffect at a substantially constant level; however, the stereoscopiceffect is varied while the zoom lens is moved because of two factors:(1) the variation of the stereoscopic effect due to variation of theincident light flux, and (2) the variation of the stereoscopic effectdue to variation of the angle of view.

In the case in which the number of the driving steps of the diaphragm16′ becomes smaller than the number of the driving steps of the zoomlens, thereby the slight variation of the stereoscopic effect isinevitable, the third embodiment of the presently disclosed subjectmatter carries out a virtual zooming using a digital zoom, therebyreducing the variation of the stereoscopic effect, instead of moving thezoom lens after the aperture of the diaphragm 16′ is reduced by onestop. Hereinafter, description will be provided on a monocularstereoscopic imaging device 3 according to the third embodiment. Theoverall configuration of the imaging device is same as that of the firstembodiment; therefore, description thereof will be omitted, and only theoperation of the imaging device will now be described. In thedescription of the operation of the imaging device, the same elements asthose of the first embodiment are referred to by the same referencenumerals and characters, and description thereof will be omitted.

[Description of Operation of Imaging Device]

Hereinafter, description will be provided on the operation of themonocular stereoscopic imaging device 3. This imaging processing iscontrolled by the CPU 40. A program for allowing the CPU 40 to executethis imaging processing is stored on the program storage unit in the CPU40.

FIG. 14 is a flow chart illustrating a flow of the photographing anddisplaying processing of the live view images in the monocularstereoscopic imaging device 3.

In response to an input of an instruction for staring the photographing(step S10), the CPU 40 drives the taking lens 14 and the diaphragm 16′to move to an initial position, so that the light of the object havingpassed through the taking lens 14 is imaged through the diaphragm 16′ onthe light receiving surface of the phase-difference CCD 17, and theleft-eye image data and the right-eye image data are generated by turns.Brightness/color difference signals are generated from the abovegenerated left-eye image data and the right-eye image, and the generatedsignals are then output to the liquid crystal monitor 28 through thevideo encoder 66, thereby displaying the 3D through images on the liquidcrystal monitor 28 (step S11).

The CPU 40 determines whether or not the zoom operation has been carriedout, that is, the instruction for the zoom operation has been inputthrough the operating unit 48 (step S12). When no instruction for thezoom operation has been input (NO in step S12), the CPU 40 shifts theprocess to step S14.

When the instruction for the zoom operation has been input (YES in stepS12), the CPU 40 moves the zoom lens through the lens driving unit 47until the focus distance becomes equal to the focus distance input asthe instruction of zooming. The CPU 40 determines whether or not a nextdestination of this zoom lens in movement is a position where the focusdistance falls under any of positions a, b, c, d, and e in FIG. 15 (stepS30).

FIG. 15 is a graph illustrating a relation among the stereoscopiceffect, the focus distance, and the driving timing of the diaphragm 16′,in the case where the number of the driving steps of the diaphragm 16′becomes smaller than that of the zoom lens, and a slight variation ofthe stereoscopic effect becomes inevitable. The graph shape of FIG. 15is same as that in FIG. 12.

As described in the variation of the first embodiment, in FIG. 12, theCPU 40 reduces the aperture of the diaphragm 16′ by one stop at thefocus distance where the stereoscopic effect becomes equal to thestereoscopic effect located at the wide-angle end. The focus distance atany of the points a, b, c, d, and e is a focus distance where thestereoscopic effect becomes equal to the stereoscopic effect located atthe wide-angle end.

When the zoom lens is moved to a position where the focus distance isnot located at any of the points a, b, c, d, and e in FIG. 15 (NO instep S30), that is, located between the points a and b, between thepoints b and c, between the points c and d, or between the points d ande, this is not the timing to drive the diaphragm 16′ (see the alternatelong and short dash lines in FIG. 15). In such a case, the CPU 40 cutsoff from the right-eye image data and the left-eye image datapredetermined areas having the same angle of view as the angle of viewwhen the zoom lens is moved, and carries out a digital zoom processingto virtually change the focus distance (step S33). In the digital zoomprocessing, there is no movement of the zoom lens (including theaccompanying movement of the focus lens), and thus no variation of thestereoscopic effect is caused by the variation of the incident lightflux. The digital zoom processing is a well-known technique, andtherefore, the description thereof will be omitted.

As similar to step S11, the areas cut off from the right-eye image dataand the left-eye image data in step S33 are three-dimensionallydisplayed in real time on the liquid crystal monitor 28 (step S34).

When the zoom lens is moved to a position where the focus distance islocated at any of the points a, b, c, d, and e in FIG. 15 (YES in stepS30), the CPU 40 moves the zoom lens, as well as drives the diaphragm16′ (step S31). When the zoom lens is moved to a position where thefocus distance is located between the points a and b, between the pointsb and c, between the points c and d, or between the points d and e, thedigital zoom processing is executed (step S33), therefore, the zoom lensis not actually moved. In step S31, when the zoom lens is located at aposition where the focus distance is located at the point a, the zoomlens is moved to a position where the focus distance is located at thepoint b, as well as the aperture of the diaphragm 16′ is reduced by onestop. Similarly, when the zoom lens is located at a position where thefocus distance is located at the point b, the zoom lens is moved to aposition where the focus distance is located at the point c, as well asthe aperture of the diaphragm 16′ is reduced by one stop; when the zoomlens is located at a position where the focus distance is located at thepoint c, the zoom lens is moved to a position where the focus distanceis located at the point d, as well as the aperture of the diaphragm 16′is reduced by one stop; and when the zoom lens is located at a positionwhere the focus distance is located at the point d, the zoom lens ismoved to a position where the focus distance is located at the point e,as well as the aperture of the diaphragm 16′ is reduced by one stop.

The CPU 40 three-dimensionally displays in real time on the liquidcrystal monitor 28 the right-eye image data and the left-eye image dataphotographed in the state after the zoom lens is moved in the samemanner as that in step S11 (step S32).

The processes from step S31 to step S34 have been explained by using theexample in which the zooming is executed in the direction from thewide-angle end toward the telephoto end, that is, the zoom lens is movedin the direction of increasing the focus distance; but these processesare also executed when the zoom lens is moved from the telephoto endtoward the wide-angle end (in the direction of decreasing the focusdistance). Specifically, when the zoom lens is moved at a position wherethe focus distance is located between the points a and b, between thepoints b and c, between the points c and d, or between the points d ande, the digital zoom processing is executed (step S33), and the imageafter the digital zoom processing is carried out is displayed (stepS34).

When the zoom lens is moved to a position where the focus distance islocated at any of the points a, b, c, d, and e in FIG. 15 (YES in stepS30), the CPU 40 moves the zoom lens as well as drives the diaphragm 16′(step S31), and displays the photographed image (step S32).Specifically, when the zoom lens is located at a position where thefocus distance is located at the point e, the zoom lens is moved to aposition where the focus distance is located at the point d, as well asthe aperture of the diaphragm 16′ is increased by one stop; when thezoom lens is located at a position where the focus distance is locatedat the point d, the zoom lens is moved to a position where the focusdistance is located at the point c, as well as the aperture of thediaphragm 16′ is increased by one stop; when the zoom lens is located ata position where the focus distance is located at the point c, the zoomlens is moved to a position where the focus distance is located at thepoint b, as well as the aperture of the diaphragm 16′ is increased byone stop; and when the zoom lens is located at a position where thefocus distance is located at the point b, the zoom lens is moved to aposition where the focus distance is located at the point a, as well asthe aperture of the diaphragm 16′ is increased by one stop (step S31).

As a result from the execution of step S31 to step S34, the CPU 40determines whether or not the zooming has been carried out until thefocus distance becomes equal to the focus distance input as theinstruction of the zooming (step S35). When the zooming is not completedyet (NO in step S35), the processes from the S30 to the S35 are executedonce again. The processes from step S31 to step S35 are repetitivelyexecuted in this manner, thereby carrying out the zooming at theposition defined as instructed.

When the zooming is completed (YES in step S35), the CPU 40 determineswhether or not an instruction for completing the photographing has beeninput (step S14). When no instruction for completing the photographinghas been input (NO in step S14), the CPU 40 executes the processes fromstep S12 to step S35 once again, and when the instruction for completingthe photographing has been input (YES in step S14), the CPU 40 completesthe photographing processing of the through images. The process for thecase in which S1 and S2 are switched ON is same as that in the firstembodiment, and therefore, description thereof will be omitted.

According to the present embodiment, of two factors that affect thevariation of the stereoscopic effect: (1) the variation of thestereoscopic effect due to variation of the incident light flux; and (2)the variation of the stereoscopic effect due to variation of the angleof view, it is possible to eliminate the factor (1). Consequently, inthe case where the number of the driving steps of the diaphragm issmaller than that of the zoom lens, and thereby the slight variation ofthe stereoscopic effect becomes inevitable, it is possible to reduce anuncomfortable feeling of the photographer.

The present embodiment has been described by using the example of thephotographing and displaying of the live view images, but thisembodiment may be applicable to the case of photographing moving images,for example, when the right-eye image data and the left-eye image dataare continuously acquired, as similar to the first embodiment.

The first to third embodiments have been described by using the exampleof employing the CCD as the image sensor, but the presently disclosedsubject matter is not limited to this. The presently disclosed subjectmatter may be applicable to other image sensors such as a CMOS(complementary metal-oxide semiconductor), or the like.

The first to third embodiments have been explained by using the exampleof the monocular stereoscopic imaging device for dividing the light fluxwith the light shielding members 17A, 17B provided on the microlens Lside of the phase-difference CCD 17, but the presently disclosed subjectmatter may be applicable to a monocular stereoscopic imaging deviceusing a taking lens 12′ including a relay lens for dividing the lightflux. The presently disclosed subject matter may also be applicable tosuch an imaging device that uses a single microlens for two types ofpixels (main pixels and subpixels), thereby limiting the incident lightflux from entering each type of the pixels.

What is claimed is:
 1. A monocular stereoscopic imaging devicecomprising: an imaging optical system including a taking lens includinga zoom lens, and a diaphragm which forms a single aperture; a singleimaging unit configured to continuously acquire a first image and asecond image based on image signals output from an imaging sensorincluding a first pixel group and a second pixel group on which an imageof a subject passed through a first area of the taking lens and an imageof the subject passed through a second area of the taking lens areformed respectively; a diaphragm driving unit configured to change adegree of an aperture of the single aperture of the diaphragm; and acontrolling unit configured to acquire an instruction of changing afocus distance, configured to drive the diaphragm through the diaphragmdriving unit so as to increase the degree of the aperture when the zoomlens is moved in a direction of decreasing the focus distance inaccordance with the acquired instruction of changing the focus distance.2. The monocular stereoscopic imaging device according to claim 1,wherein the controlling unit drives the diaphragm through the diaphragmdriving unit so as to reduce the degree of the aperture when the zoomlens is moved in a direction of increasing the focus distance.
 3. Themonocular stereoscopic imaging device according to claim 1, wherein thecontrolling unit controls the diaphragm driving unit to minimize thedegree of the aperture when the zoom lens is located at a telephoto end.4. The monocular stereoscopic imaging device according to claim 1,wherein the controlling unit controls the diaphragm driving unit tomaximize the degree of the aperture when the zoom lens is located at awide-angle end.
 5. The monocular stereoscopic imaging device accordingto claim 1, wherein the diaphragm can be driven so as to change thedegree of the aperture in n steps (n is a natural number equal to orgreater than 2), the zoom lens can be driven so as to change the focusdistance in m steps (m is a natural number equal to or greater than 2),m being greater than the n, and the controlling unit performs control toreduce the aperture of the diaphragm by one step before the zoom lens ismoved when the instruction of changing the focus distance in thedirection of increasing the focus distance is input through an inputunit, and to reduce the aperture of the diaphragm by one step every timethe zoom lens is moved by a predetermined number of steps.
 6. Themonocular stereoscopic imaging device according to claim 5, wherein thecontrolling unit limits the driving steps of the zoom lens to be the nsteps of the m steps, and performs control to synchronously drive thediaphragm and the zoom lens.
 7. The monocular stereoscopic imagingdevice according to claim 5, further comprising a digital zoom unitconfigured to cut off predetermined areas from the acquired images, andconfigured to electronically change the focus distance, wherein thecontrolling unit controls the digital zoom unit to virtually change thefocus distance through the digital zoom unit, instead of moving the zoomlens.
 8. The monocular stereoscopic imaging device according to claim 1,further comprising a storage unit configured to store a relation betweenthe degree of the aperture of the diaphragm and the focus distance,wherein the controlling unit controls the diaphragm driving unit basedon the relation between the degree of the aperture and the focusdistance stored on the storage unit.
 9. The monocular stereoscopicimaging device according to claim 1, further comprising: atwo-dimensional image generating unit configured to synthesize theacquired images when brightnesses of the acquired images are equal to orless than a predetermined value, so as to generate a two-dimensionalimage; a display unit configured to display the two-dimensional image;and a display controlling unit configured to display the generatedtwo-dimensional image on the display unit when the two-dimensional imageis generated by the two-dimensional image generating unit.
 10. Themonocular stereoscopic imaging device according to claim 1, furthercomprising: a display unit configured to recognizably display theacquired images as a three-dimensional image; and a display controllingunit configured to three-dimensionally display on the display unit theimages continuously acquired.
 11. A method of controlling a monocularstereoscopic imaging device comprising: an imaging optical systemincluding a taking lens including a zoom lens, and a diaphragm whichforms a single aperture; a single imaging unit configured tocontinuously acquire a first image and a second image based on imagesignals output from an imaging sensor including a first pixel group anda second pixel group on which an image of a subject passed through afirst area of the taking lens and an image of the subject passed througha second area of the taking lens are formed respectively; and adiaphragm driving unit configured to change a degree of an aperture ofthe single aperture of the diaphragm, the method comprising: acquiringan instruction of changing a focus distance; and driving the diaphragmthrough the diaphragm driving unit so as to increase the degree of theaperture when the zoom lens is moved in a direction of decreasing thefocus distance in accordance with the acquired instruction of changingthe focus distance.
 12. The method according to claim 11, furthercomprising driving the diaphragm through the diaphragm driving unit soas to reduce the degree of the aperture when the zoom lens is moved in adirection of increasing the focus distance.